Higher-Order Theory of Mind in Negotiations under Incomplete Information

Theory of mind refers to the ability to reason explicitly about unobservable mental content such as beliefs, desires, and intentions of others. People are known to make use of theory of mind, and even reason about what other people believe about their beliefs. Although it is unknown why such a higher-order theory of mind evolved in humans, exposure to mixed-motive situations may have facilitated its emergence. In such mixed-motive situations, interacting parties have partially overlapping goals, so that both competition and cooperation play a role. In this paper, we consider negotiation using alternative offers in a particular mixed-motive situation known as Colored Trails, and determine to what extent higher-order theory of mind is beneficial to computational agents. Our results show limited effectiveness of first-order theory of mind, while second-order theory of mind turns out to benefit agents greatly by allowing them to reason about the way they communicate their interests.